Large Language Models (LLMs) have been positioned as having the potential to expand access to health information in the Global South, yet their evaluation remains heavily dependent on benchmarks designed around Western norms. We present insights from a preliminary benchmarking exercise with a chatbot for sexual and reproductive health (SRH) for an underserved community in India. We evaluated using HealthBench, a benchmark for conversational health models by OpenAI. We extracted 637 SRH queries from the dataset and evaluated on the 330 single-turn conversations. Responses were evaluated using HealthBench's rubric-based automated grader, which rated responses consistently low. However, qualitative analysis by trained annotators and public health experts revealed that many responses were actually culturally appropriate and medically accurate. We highlight recurring issues, particularly a Western bias, such as for legal framing and norms (e.g., breastfeeding in public), diet assumptions (e.g., fish safe to eat during pregnancy), and costs (e.g., insurance models). Our findings demonstrate the limitations of current benchmarks in capturing the effectiveness of systems built for different cultural and healthcare contexts. We argue for the development of culturally adaptive evaluation frameworks that meet quality standards while recognizing needs of diverse populations.

Hertility is a women's health company built by women, for women. We're shaping the future of reproductive healthcare by pioneering unique diagnostic testing that provides data-driven and advanced insights into reproductive health, fertility decline and the onset of menopause. We provide expert advice, education and access to care - all from the comfort of your home. We tailor pathways to the individual's reproductive goals, whether it be to explore their ovarian health and fertility options or overall hormonal health, such as confirming a PCOS diagnosis. Ultimately, our aim is to change attitudes around reproductive health, both for individuals and in the workplace, and to encourage women to be proactive by tracking their reproductive health.

An innovative chatbot designed for sharing critical information about sexual and reproductive health (SRH) with young people in India is demonstrating how artificial intelligence (AI) applications can engage vulnerable and hard-to-reach population segments. Working with the Population Foundation of India (PFI), Helen Wang, associate professor of communication, College of Arts and Sciences, examined the user-centered design and engagement of SnehAI, the first Hinglish (Hindi and English) chatbot purposefully developed for social and behavioral change. "Many AI technologies today are motivated by profit, but we must also be aware that AI can be leveraged in ways that facilitate social and behavior change," says Wang, who specializes in entertainment-education and storytelling as instruments for health promotion. "SnehAI is a powerful testimonial of the vital potential that lies in AI for good." The findings from Wang's instrumental case study appear in the Journal of Medical Internet Research.

A technician took X-rays of Gura's chest but couldn't make heads or tails of the resulting images. "We had to wait a few days for the radiologist to come from the big city to diagnose what I had," delaying his treatment, Gura says. "I asked: How come we don't have a centralized reference database of all the X-rays for people like me so that I can just run a computer vision comparison against it and get my own sense of what's going on in my body? That was the seed of the idea." The idea became Zebra Medical Vision, which transforms vast amounts of medical imaging data into actionable insights, allowing doctors to better detect diseases, tumors and fractures while giving patients more information about their health.

There has been little innovation in the analysis of male fertility since the 1950s. Why? Tobias Boecker, Paris-based medtech entrepreneur is co-founder of Nanovare – a system that uses AI to help medical professionals to improve and speed up male fertility analysis – explores the current crisis in this field and what can be done to change things Birth rates are dropping, notably in the Western world. Between 1960 and 2015, the birth rate per woman in Germany decreased from 2.4 to 1.5, in the UK from 2.7 to 1.8 and in the US from 3.7 to 1.8. There are many external factors that influence this trend such as the increased age of marriage, the focus on professional careers, or the modernisation of social systems which eliminate parents' need to have many children to help support them in old age. Another factor that is becoming increasingly important is our own reproductive health, which depends on our lifestyle choices (eg smoking), environmental influences (eg pollution), and medical factors (eg genetic preconditions).

When I first told to one of my girlfriends that our chatbot would do period tracking, her smile disappeared from her face. She asked why would anybody still do period tracking anywhere in the era of so many other applications that focus on women and their reproductive health. Well, I think the market is still underserved and there are still many opportunities, and one leads to creating a mix of a friend and a virtual gynaecologist using AI. Let's face it: we are absolutely not living in an equal world. As much as we fought to have this freedom and rights, we still have a lot to do.